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[FY02-033]  -  Developing  a  detailed  integrated  optics  computational  model  capable 
of  quantitative  descriptions  and  predictions  of  the  image  analysis  performance  of 
compound  eyes 

Doekele  G.  Stavenga 

Department  of  Neurobiophysics,  University  of  Groningen,  the  Netherlands 

Summary 

This  is  the  final  report  of  a  three-year  project  supported  by  the  EOARD.  The  aim  of  the 
project  has  been  to  contribute  to  a  quantitative  description  of  image  processing  by 
compound  eyes.  The  main  results  of  the  performed  research  on  three  types  of  compound 
eyes  are  summarized,  and  a  list  of  publications  is  appended. 

Introduction 

Compound  eyes  are  the  imaging  systems  of  insects  and  crustaceans.  These  compactly 
built  animal  machines  fulfill  a  variety  of  tasks  essential  for  survival:  find  food,  find 
mates,  and  avoid  predators.  Arthropods  have  developed  many  potentially  useful  sensors, 
which  could  be  considered  revolutionary  if  they  were  engineered  for  use  in  autonomous 
robots.  The  project  is  part  of  a  long-term  initiative  to  develop  an  advanced  optical  sensor, 
that  is  a  multi-aperture  seeker,  motivated  by  detailed  understanding  of  the  arthropod 
(insect)  compound  eye. 

From  an  optics  point  of  view,  flies  have  arguably  the  simplest  compound  eye.  The 
fly  eye  is  now  well  understood,  notably  due  to  the  recent  work  performed  in  the  early 
phases  of  the  project.  This  knowledge  base  has  provided  a  solid  foundation  for  the  study 
of  the  more  complicated  compound  eyes  of  other  invertebrate  species  to  which  we  have 
shifted  subsequently.  For  studying  the  simultaneous  processing  of  spatial  and  spectral 
information,  butterflies  have  great  potential.  This  is  firstly  because  of  their  large  and 
accessible  eyes,  and  secondly  because  they  have  developed  a  wide  range  of  optical  tools 
to  achieve  sophisticated  image  acquisition.  Various  butterfly  species  have  most  exquisite 
eyes  with  a  vast  structural  as  well  as  functional  diversity.  Our  present  research  is  mostly 
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concentrated  on  combined  experimental  and  theoretical  studies  of  butterfly  eyes,  so  to 
develop  and  test  quantitative  models  of  visual  imaging  and  infonnation  processing. 

Results  achieved  during  the  project  period 

During  the  project  period  important  progress  has  been  realized  by  studying  three  different 
compound  eye  types:  /)  the  neural  superposition  eyes  of  flies,  ii)  the  apposition  eyes  of 
butterflies  and  crabs,  and  iii)  the  optical  superposition  eyes  of  moths  and  other  insects 
and  crustaceans. 

Compound  eyes  are  composed  of  numerous  (several  thousand)  eyelets,  so-called 
ommatidia,  each  having  a  facet  lens  and  a  crystalline  cone,  which  together  fonn  the 
imaging  dioptrics,  and  8  or  9  photoreceptor  cells,  which  are  surrounded  by  screening 
pigment  cells  functioning  as  optical  barriers  for  stray  light.  The  light-sensitive  apparatus 
of  a  photoreceptor  cell,  formed  by  the  visual  pigment  molecules  and  the 
phototransduction  machinery,  which  transforms  absorbed  light  into  an  electrical  signal,  is 
packed  in  a  special  organelle,  the  rhabdomere.  The  set  of  rhabdomeres  of  an  ommatidium 
is  called  the  rhabdom.  In  fly  eyes,  the  rhabdomeres  are  spatially  separate  cylinders,  each 
acting  as  an  optical  waveguide.  In  bee  and  butterfly  eyes,  the  rhabdomeres  are  joined  into 
a  fused  cylinder,  which  thus  acts  as  one  functional  optical  waveguide. 

The  basic  architecture  of  compound  eyes  is  recognized  in  the  apposition  eye  type, 
where  each  facet  lens  focuses  incident  light  from  a  narrow  spatial  field  into  the  fused 
rhabdom.  In  optical  superposition  eyes  a  large  set  (often  several  hundred)  of  facet  lenses 
together  relay  light  into  one  rhabdom.  This  occurs  due  to  the  special  optics  of  the 
crystalline  cones,  which  act  as  so-called  lens  cylinders,  akin  to  SELFOC  optical  fibers, 
and  the  presence  of  a  clear  zone  between  the  dioptrics  and  the  rhabdom  layer.  In  neural 
superposition  eyes,  an  individual  facet  lens  channels  light  into  a  number  of  spatially 
separate  rhabdomeres,  each  of  which  receives  light  from  a  slightly  different  visual  field. 
The  different  optics  of  the  three  compound  eye  types  are  reflected  in  different  anatomical 
arrangements.  Each  of  the  specific  eye  types,  as  a  front  end  of  a  sensitive  imaging 
system,  has  certain  advantages  and  disadvantages,  which  must  be  understood  before 
attempting  technical  applications. 
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Modeling  the  integrated  optics  of  fly  eyes 

The  optical  characteristics  of  the  neural  superposition  eyes  of  flies  have  been  analyzed  in 
the  first  phase  of  the  research  period.  A  comprehensive  wave-optical  model  has  been 
developed,  which  was  possible  owing  to  extensive  knowledge  previously  acquired  in 
experimental  optical  as  well  as  electrophysiological  studies  [1,2, 5, 6;  the  numbers  in 
brackets  refer  to  the  list  of  publications  at  the  end  of  this  report].  The  model  includes  the 
diffraction  properties  of  the  facet  lenses,  the  waveguide  properties  of  the  rhabdomeres, 
and  the  spectral  dependences  of  the  absorption  coefficient  of  the  visual  pigments.  It  has 
been  demonstrated  that  the  model  quantitatively  and  accurately  describes  experimentally 
measured  sensitivity  spectra  as  well  as  angular  sensitivities  of  fly  photoreceptors  [1,5]. 
The  model  thus  constituted  a  firm  basis  for  the  analysis  of  image  processing  by  the  set  of 
photoreceptors  in  a  fly  compound  eye.  For  instance,  the  model  has  provided  insight  into 
the  robust  optics  of  the  integrated  optical  system  of  a  fly  facet  lens  and  the  rhabdomere 
optical  waveguide,  where  the  diameter  of  a  facet  lens  is  20-30  pm  and  that  of  the 
waveguide  is  1-2  pin.  The  model  showed  that  it  is  rather  uncritical  where  the  entrance 
plane  of  the  waveguide  is  positioned.  It  was  generally  assumed  before  that  the  tip  of  the 
waveguide  should  be  precisely  placed  in  the  focal  plane  of  the  lens  (focal  distance  ca  60 
pm),  but  it  appeared  that  the  wave  properties  of  light  allow  considerable  play,  some  10- 
20  pm,  for  maximal  capture  of  incident  light  [1], 

The  model  furthermore  offered  an  explanation  for  some  extraordinary  optical 
properties  of  the  compound  eyes  of  the  fruitfly  Drosophila ,  perhaps  the  most  intensely 
studied  biological  system  [2],  This  tiny  fly  has  extremely  small  facet  lenses  with 
surprisingly  short  focal  distances,  but  it  still  manages  to  see  well.  This  is  realized  by 
accepting  a  slight  loss  in  spatial  resolution,  while  compressing  the  minute  optical  system 
into  a  very  small  space  [2].  It  is  now  also  understood  how  the  pupil  mechanism,  a  light 
control  system  mediated  by  movable  pigment  granules,  optimizes  the  angular  sensitivity 
of  the  photoreceptors,  depending  on  the  average  light  intensity  in  the  environment  [5]. 

The  model  has  in  addition  enabled  the  assessment  of  the  sensitivity  enhancement 
provided  by  the  carotenoid  derivative  3-hydroxyretinal,  which  acts  as  a  unique  antenna  or 
sensitizing  pigment  in  fly  eyes  [6],  In  conclusion,  we  now  have  gained  a  quite  complete 
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understanding  of  the  optical  factors  that  determine  the  spectral  properties  as  well  as  the 
spatial  acuity  of  fly  eyes. 

Optics  of  butterfly  eyes 

Subsequently  to  the  fly  eye,  the  optical  and  physiological  characteristics  of  butterfly 
apposition  eyes  have  been  studied.  The  optics  of  these  eyes  is  considerable  more 
complicated  than  the  optics  of  fly  eyes.  Firstly,  the  eye’s  visual  waveguide,  the  rhabdom, 
contains  at  least  three  different  types  of  visual  pigment,  and  secondly,  the  crystalline 
cone,  which  forms  together  with  the  facet  lens  the  dioptric  system  of  an  ommatidium,  has 
a  very  concentrated  gradient  refractive  index,  and  thirdly,  there  is  a  so-called  tapetum 
below  the  waveguide,  which  reflects  part  of  the  propagated  light  back  into  the 
waveguide;  this  all  makes  an  optical  analysis  quite  difficult.  Nevertheless,  we  have  been 
able  to  develop  a  working  computational  model  for  the  butterfly  rhabdom.  By  adapting 
the  model  to  specific  butterfly  cases,  it  could  quantitatively  describe  the  spectral 
sensitivities  measured  from  different  classes  of  photoreceptors  in  the  papilionid  Papilio 
xuthus  as  well  as  the  cabbage  butterfly  Pieris  rapae  [7,  9,  10,  14].  Using  essentially  the 
same  model,  measured  time  courses  of  absorbance  spectra  that  occur  during  light-induced 
photochemical  conversions  of  the  visual  pigments  in  the  butterfly  Polygonia  c-album,  an 
exemplary  case  of  nymphalid  butterflies,  were  described,  yielding  the  absorption  spectra 
of  the  visual  pigments  and  their  photoproducts  [4,  12]. 

Butterfly  eyes  are  specifically  challenging  because  several  different  optical 
methods  are  utilized  to  realize  color  discrimination.  That  at  least  is  the  working 
hypothesis  resulting  from  several  studies  on  butterfly  vision.  The  papilionid  butterflies 
have  two  types  of  spectral  filters,  realized  by  yellow-  and  red-colored  pigments,  that  are 
located  in  clusters  of  granules  surrounding  the  rhabdom  waveguides.  They  shift  the 
spectral  sensitivities  to  longer  wavelengths  as  well  as  create  narrow  spectral  bandwidths 
[10].  This  results,  among  others,  in  sensitivity  spectra  for  a  set  of  photoreceptors  peaking 
in  the  red,  which  presumably  enables  the  red  sensitivity  necessary  for  the  butterflies  to 
detect  red  flowers  [10,  17].  Cabbage  butterflies  have  elaborated  the  tuning  of  red 
receptors  by  installing  two  types  of  red  pigment  filters,  resulting  in  two  classes  of  red 
receptor  with  very  narrow  bandwidths,  peaking  in  the  red  and  deep-red  wavelength  range 
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[9],  Cabbage  butterflies  most  likely  also  have  an  extremely  sophisticated  spectral 
discrimination  system  in  the  short  wavelength  range,  as  they  exploit  three  classes  of 
visual  pigment,  causing  photoreceptors  with  maximal  absorption  in  the  UV,  violet  and 
blue;  in  virtually  all  other  arthropods  there  are  only  two  short-wavelength  classes,  UV 
and  blue.  Male  cabbage  butterflies  have  even  pushed  the  variation  in  receptor  type 
further,  namely  by  modifying  the  violet  receptor  into  a  double-peaked  blue  receptor.  This 
is  achieved  by  recruiting  a  special  blue  filter  [14]. 

The  research  on  color  receptor  types  has  strongly  stimulated  an  additional  line  of 
research,  namely  that  on  coloration  methods  of  insect  bodies,  specifically  of  butterfly 
wings  [8,  13,  15,  17].  Measurements  on  numerous  butterfly  species  indicate  that  the 
sensitivity  spectra  of  the  visual  photoreceptors  are  tuned  to  the  wing  reflectance  spectra. 
This  finding  indicates  that  studies  on  sensors  and  their  technical  applications  should  be 
integrated  with  studies  that  characterize  the  objects  that  must  be  detected. 

Superposition  eyes  and  their  delicacies 

The  optical  superposition  eye,  the  eye  type  utilized  by  nocturnal  moths,  has  a  special 
design.  Each  photoreceptor  receives  light  via  a  set  of  multiple  facets,  which  endows  these 
eyes  with  a  photosensitivity  that  is  several  log  units  higher  than  that  of  apposition  eyes, 
because  in  the  latter  case  only  one  facet  lens  channels  light  into  a  rhabdom.  Many 
nocturnal  or  crepuscular  active  arthropods  therefore  use  superposition  eyes.  The  problem 
with  this  eye  design  is  however  that  the  acceptance  angles  of  the  photoreceptors  are 
easily  distinctly  broadened  compared  to  the  acceptance  angles  of  photoreceptors  in 
apposition  eyes.  The  proper  focusing  by  a  large  set  of  dioptrical  systems  onto  one 
rhabdom  appears  to  be  a  tall  order  for  the  builders  of  real  superposition  eyes.  The 
developmental  process,  which  is  of  course  under  genetic  control,  must  very  precisely 
construct  the  crystalline  cones  and  their  refractive  gradients  in  order  to  have  perfect 
focusing  optics.  That  appears  not  to  be  realized  in  most  eyes,  presumably  because  the 
necessary  genetic  effort  would  mean  major  investments  for  which  there  are  inadequate 
returns.  To  understand  this,  one  must  consider  that  nocturnal  animals  have  access  to  little 
light  and  thus  must  rely  on  temporal  as  well  spatial  integration  (so-called  pooling)  to 
achieve  a  sufficient  signal  to  noise  ratio  with  their  photoreceptors.  Nocturnal  eyes 
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therefore  have  photoreceptors  with  wide  acceptance  angles.  Nevertheless  a  number  of 
day-active  moths  with  superposition  eyes  exist.  Not  only  is  their  light  sensitivity  high,  but 
their  spatial  resolution  appears  to  be  high  as  well. 

In  order  to  learn  more  about  the  potential  imaging  properties  of  optical 
superposition,  as  well  as  of  their  limitations,  we  have  recently  perfonned  a  number  of 
optical  studies  on  superposition  eyes.  This  work  has  revealed  some  important 
misconceptions  in  the  literature.  Firstly,  the  existing  geometric  optical  treatments  have 
been  incomplete,  and  they  have  failed  to  realize  that  superposition  eyes  essentially  act  as 
negative  refractive  index  materials.  A  general  optical  framework  of  superposition  eyes 
has  been  formulated  [20].  Secondly,  superposition  eyes  are  assumed  to  be  essentially 
limited  by  single  facet  lens  diffraction,  because  of  incoherence.  A  closer  look  shows  that 
the  studies  that  led  to  this  concept  have  been  inadequate.  The  conclusion  instead  is  that 
partial  coherence  is  a  crucial  factor  in  superposition  eye  imaging  and  that  the  only 
important  source  of  imaging  errors  are  the  optical  imperfections  of  the  dioptrical 
elements:  facet  lens  and  crystalline  cone  [21]. 

Moth  eyes  are  special  in  that  the  outer  surface  of  the  facet  lenses  is  studded  with 
nanosized  protuberances,  which  together  act  as  an  impedance  matching  device.  This  so- 
called  corneal  nipple  array  (CNA)  was  discovered  40  years  ago.  (The  CNA  research 
received  essential  support  from  AF  EOAR.)  This  work  has  since  led  to  numerous 
technical  applications,  known  as  moth-eye  devices,  including  the  anti-radar  stealth 
technology.  We  have  developed  an  optical  model  that  quantitatively  estimates  the  effect 
of  the  comeal  nipple  array  on  reflectance  [13,  16,  21].  Calculations  show  that  the  height 
of  the  nipples  is  the  crucial  factor  that  detennines  the  surface  reflectance.  The  benefit  of 
the  comeal  nipple  array  for  photosensitivity  enhancement  appears  to  be  minor,  but  the 
array  is  extremely  effective  for  reducing  the  reflectance  and  hence  the  visibility  of  the 
eyes  and  their  owner. 

Radiation  of  the  supported  research 

Other  investigators  of  compound  eyes  now  use  the  developed  wave-optical  model  of  the 
integrated  fly  facet-lens  and  optical-waveguide  system  in  their  research.  A  notable 
example  is  the  work  by  Lee  and  co-workers.  They  have  created  artificial  compound  eyes 
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with  recon  figurable  microtemplating,  that  is  polymer  replication  of  microlens  patterns 
and  formation  of  polymer  waveguides,  self-aligned  with  the  microlenses  via  a  self¬ 
writing  process  in  photosensitive  polymer  resin  (Lee  LP,  Szema  R:  Inspirations  from 
biological  optics  for  advanced  photonic  systems,  Science  310:1 148-1 150,  2005;  Kim  JY, 
Jeong  KH,  Lee  LP:  Artificial  ommatidia  by  self-aligned  microlenses  and  waveguides, 
Optics  Letters  30:  5-7,  2005).  The  research  of  the  Lee  team  is  a  shining  example  of 
biomimetics  that  was  inspired  by  both  experimental  and  theoretical  biophysical  studies 
on  compound  eyes.  It  may  be  safely  predicted  that  this  line  of  applied  science  will  further 
benefit  from  studies  on  real  visual  systems,  in  this  case  insect  eyes. 

The  fundamental  work  on  both  compound  eye  vision  and  insect  coloration,  which 
has  been  supported  by  the  EOARD,  has  become  recognized  by  the  community  of  insect 
vision  investigators,  as  is  demonstrated  by  the  regular  invitation  to  contribute  reviews  of 
this  research  for  books  and  for  scientific  as  well  as  popular  journals  [7,  10,  11,  13,  15,  17, 
22],  during  the  project  period. 

Further  activities 

Ric  Wehling  (from  Eglin  AFB)  and  Valerie  Martindale  (EOARD)  made  a  site  visit  at  4 
April  2005.  Ensuing  Ric  and  I  visited  the  Zoology  department  of  Cambridge  University 
(Prof  Simon  Laughlin  and  Dr  Jeremy  Niven)  where  we  gave  talks  and  had  discussions  on 
Vision  inspired  machines  and  optical  flight  control. 

I  participated  in  a  WOS  meeting  in  Tucson,  hosted  by  Ric  Wehling,  where  I 
lectured  on  our  work  on  ‘Compound  eye  modeling’  (13-15  April  2005). 

I  then  visited  Eglin  AFB,  and  I  delivered  three  lectures  there: 

-  ‘Insect  compound  eyes:  sophisticated  and  ultralight  visual  systems’, 

-  ‘Understanding  the  insect  retina  by  quantitative  modeling:  spatial  and  spectral 
properties  of  fly  and  butterfly  eyes’, 

-  ‘Photonic  crystals  on  the  wing:  sexy  structural  colorations  in  butterflies,  beetles  and 
damselflies’. 


7 


I  delivered  lectures  related  with  the  research  reported  here  at  a  number  of  conferences 
and  symposia,  of  which  the  most  relevant  are: 

-  Annual  Meeting  Finnish  Physical  Society  (invited  by  Prof.  M.  Weckstrom),  Oulu, 
March  2004, 

-  International  Conference  of  Entomology,  Brisbane,  August  2004;  I  organized  a  session 
on  Insect  Vision,  together  with  Prof  E.  Warrant  (University  of  Lund), 

-  Symposium  Insect  Sensors  and  Robotics  (invited  by  Prof.  M.V.  Srinivasan),  Brisbane, 
August  2004, 

-  Workshop  Interface  Disorder  in  Nanosystems  (invited  by  Prof.  G.  Palasantzas),  Leiden, 
June  2005, 

-  C.N.R.S.  -  Jacques  Monod  Conference,  Physico-Chemical  Ecology  Of  Organisms 
(invited  by  Prof.  J.  Casas),  Roscoff,  September  2005. 

Appointments 

This  research  has  been  performed  in  collaboration  with  a  number  of  students  and  two 
appointed  researchers.  A  postdoc,  Dr  K.J.A.  Vanhoutte,  performed  in  2003  a  3  months 
research  period,  when  he  worked  at  a  computational  imaging  analysis  program.  He  also 
finalized  a  photochemical  study  of  butterfly  visual  pigments.  Furthermore  a  PhD  student 
was  attracted.  P.  Pirih  started  in  September  2004  for  a  4  year  period,  to  perform  a  study 
on  butterfly  color  vision. 

Perspectives  and  conclusions 

The  research  reported  here  will  naturally  be  continued.  Knowledge  of  the  complicated 
interaction  of  spatial  and  spectral  elements  that  detennine  spatial  and  color  vision  of 
insects  and  crustaceans  is  still  quite  incomplete.  Further  study  is  necessary  to  pinpoint  the 
crucial  factors  for  optimal  vision  with  apposition  eyes.  Butterfly  eyes  will  stay  to  be  a 
rich  source  for  research  leading  to  novel  insights.  As  noted  above,  some  technical 
applications  have  already  been  attempted,  but  this  approach  is  still  in  its  infancy. 

The  insight  into  the  critical  points  of  superposition  eyes  that  has  recently  been 
achieved  will  quite  likely  facilitate  the  nano-optical  manufacturing  of  artificial 
superposition  compound  eyes,  as  has  been  realized  already  for  the  artificial  apposition 
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eyes.  The  support  provided  by  the  EOARD  has  been  instrumental  for  achieving  the 
results  presented  here.  The  sponsoring  is  gratefully  acknowledged. 
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